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Abstract 


Recognizing  the  need  to  succeed  in  a  new  multilateral,  asymmetric  threat  environment,  the 
U.  S.  Department  of  Defense  has  initiated  a  radical  transformation  in  operations  to  promote 
agility  and  enhance  responsiveness.  The  transformation  process,  as  well  as  the  resulting  new 
order  of  operations,  relies  heavily  on  system-of-systems  solutions  to  bridge  existing  gaps  in 
operations.  To  date,  a  pervasive,  and  possibly  detrimental,  assumption  has  dominated  the 
program  management  arena:  management  tools  and  methods  that  work  for  single  systems 
apply  equally  well  to  the  acquisition  of  system-of-systems  solutions.  This  technical  note 
questions  the  general  assumption  that  single-system  methods  are  effective  in  a  system-of- 
systems  arena.  Taking  the  position  that  the  field,  as  a  whole,  lacks  an  adequate  understanding 
of  the  unique  challenges  that  influence  system-of-systems  initiatives,  this  report  presents  a 
case  for  the  investigation  and  adaptation  of  structural  and  dynamic  modeling  techniques  to 
the  engineering  of  systems  of  systems.  The  report  also  includes  results  from  a  survey  of 
subject  matter  experts  providing  evidence  that  resource  expenditures  in  areas  important  to  a 
system-of-systems  environment  are  becoming  high  priorities. 
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1  Introduction 


“Measure  what  is  measurable,  and  make  measurable  what  is  not  so.” 

Galileo  Galilei  (1564-1642) 

The  need  for  joint  capabilities  has  stimulated  interest  in  integration  and  interoperability 
strategies.  As  such,  system-of-systems 1  solutions  represent  a  new,  and  important,  commodity 
class  in  the  acquisition  domain  [Krygiel  99].  In  terms  of  the  investment  resources  allocated  to 
them  and  the  operational  value  of  the  capabilities  they  provide,  systems  of  systems  have 
tremendous  implications  for  U.S.  Department  of  Defense  (DoD)  performance.  To  date,  a 
pervasive,  and  possibly  detrimental,  assumption  has  dominated  the  program  management 
arena:  management  tools  and  methods  that  work  for  single  systems  apply  equally  well  to  the 
acquisition  of  system-of-systems  solutions.  This  report  questions  the  general  assumption  that 
single-system  methods  are  effective  in  a  system-of-systems  arena.  Taking  the  position  that 
the  field,  as  a  whole,  lacks  adequate  understanding  of  the  unique  cost  drivers  that  influence 
system-of-systems  initiatives,  this  report  advocates  the  need  for,  and  potential  of,  the 
application  of  nontraditional  decision-support  tools  in  engineering  system-of-systems 
solutions. 

We  begin  the  report  with  an  overview  of  the  genesis  of  joint  capabilities  (Section  2).  We  then, 
in  Section  3,  relate  the  findings  of  a  survey  of  subject  matter  experts  [Brown  05a]  and  discuss 
the  implications  of  conflicts  between  the  desire  to  apply  traditional  methods  and  evidence 
that  these  methods  are  not  sufficient  to  address  the  complexity  of  system-of-systems  efforts. 
In  Section  4,  we  characterize  the  likely  implications  that  joint  capabilities  will  have  on  the 
acquisition  of  system-of-systems  solutions.  In  Section  5,  we  offer  a  research  agenda  that  calls 
for  the  application  of  tools  that  have  shown  value  in  analogous  (non-software  engineering) 
complex  domains. 


1  For  the  purposes  of  this  research,  a  system  of  systems  does  not  represent  a  particular 

implementation  method;  rather,  in  this  report,  a  system  of  systems  relates  to  a  broad  class  of 
integration  and  interoperability  strategies.  See  http://www.sei.cmu.edu/publications 
/documents/06. reports/06tr003.html  and  http://www.infoed.com/Open/PAPERS/systems.htm  for 
discussions  of  systems  of  systems. 
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2  The  Genesis  of  Joint  Capabilities2 


During  the  Cold  War  era,  military  strategy  was  predicated  on  the  belief  that  deterrence  was 
best  achieved  through  arms  superiority.  The  arms  race  was  won  by  a  heavy  reliance  on 
scientific -management  principles  as  an  organizing  paradigm  [Hughes  98].  Economies  of 
scale  were  achieved  in  arms  production  through  a  capital-intensive  industrial  base  that 
stressed  the  principles  of  scientific  management:  hierarchy,  division  of  work,  functional 
specialization,  and  the  separation  of  planning  from  operations.  These  strategies  gave  rise  to  a 
plethora  of  individual  subcultures  with  distinct  missions,  goals,  and  vocabularies. 

From  a  resource  perspective,  programs  were  defined  by  each  armed  service  unit,  collectively 
submitted  to  the  Office  of  the  Secretary  of  Defense  (OSD)  for  review  and  approval,  and  then 
incorporated  into  the  President’s  budget  [DoD  03a],  Guidance  issued  by  the  OSD  at  the 
beginning  of  the  fiscal  cycle  gave  each  of  the  armed  services  a  target  that  reflected  an 
equitable  distribution  of  resources.  Generally,  equities  were  preserved,  and  programs  would 
get  their  start  without  a  great  deal  of  scrutiny  by  the  Joint  Command. 

Traditional  system  development  was  command-centric  and  based  on  assumptions  of  known 
nation-state  threats  that  could  be  symmetrically  balanced  with  well-defined  (and  often 
overwhelming)  force  capabilities.  Systems  could  be  rigorously  specified,  developed,  tested, 
and  placed  into  operation  with  clear  separation  of  labor  and  repeatable,  parameterized  metrics 
for  cost,  schedule,  and  performance.  When  these  systems  went  awry,  management  and 
independent  cost  estimators  were  well  equipped  to  detect  and  provide  insight  into  problems 
in  a  timely  manner. 

From  a  transparency  and  accountability  perspective,  the  scientific -management  method  of 
organizing  activities  simplified  the  budgeting  process  and  facilitated  oversight.  But  it  did  so 
at  the  expense  of  the  integration  and  agility  needed  to  deter  immediate  threats.  In  the  current, 
post-Cold  War  context — where  complexity  and  variability  dominate — the  scientific- 
management  method  becomes  less  useful.  In  this  environment,  scaling  up  from  a  single 
system  to  a  system  of  systems  often  results  in  nonlinear  diseconomies  of  scale.3 

DoD  transformation  became  a  compelling  objective  in  the  aftermath  of  September  1 1 ,  200 1 . 
Military  performance  goals  now  stress  adaptive  planning,  accelerated  acquisition  cycles  built 
on  spiral  development,  output-based  management,  and  a  reformed  analytic  support  agenda 


2  This  section  is  modified  from  the  article  Joint  Capabilities  of  System-of-Systems  Solutions  [Brown 
05b]. 

3  Diseconomies  of  scale  is  an  economic  concept  referring  to  a  situation  in  which  economies  of  scale 
no  longer  function  for  an  organization.  Rather  than  experiencing  continued  decreasing  costs  per 
increase  in  output,  organizations  see  an  increase  in  marginal  cost  when  output  is  increased 

(http :// www.  investopedia.  com) . 
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[DoD  03b].  To  maintain  an  operational  advantage,  the  DoD  has  shifted  focus  from  mass  and 
firepower  to  agility  and  precision  [JV20  06].  Quite  suddenly,  agile,  tightly  integrated  joint 
operations  are  needed,  in  which  functional  specialists  are  brought  together  to  provide  a 
specific  capability  suited  to  a  particular  operational  context.  These  joint  operations  cause  a 
shift  from  command-centric  requirements  to  edge-enabled  [Alberts  03], 4  asymmetric,  user- 
demand-driven  requirements.5  Consequently,  the  broad-scale  use  of  scientific  management  as 
an  organizing  principle  has  become  suspect  [JV10  06]. 


4  The  idea  of  edge  organizations  was  introduced  by  Alberts  and  Evans  in  the  book  Power  to  the 
Edge:  Command  and  Control  in  the  Information  Age,  in  which  we  find  the  following:  “Edge 
organizations  move  senior  personnel  into  roles  that  place  them  at  the  edge.  They  often  reduce  the 
need  for  middle  management  whose  role  is  to  manage  constraints  and  control  measures.  Command 
and  control  becomes  unbundled.  Commanders  become  responsible  for  creating  initial  conditions 
that  make  success  more  likely  .  .  .”  [Alberts  03]. 

5  This  information  is  paraphrased  from  an  interview  with  B.  Cohen,  a  Carnegie  Mellon  Software 
Engineering  Institute  Visiting  Scientist,  at  the  City  University,  London  in  2005. 
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3  Evidence  that  Traditional  Methods  Are  Not 


Keeping  Pace 


We  have  observed  problems  that  DoD  organizations  have  experienced  when  they  rely  on 
traditional  methods  and  tools  for  system-of-systems  acquisition.  Witness  the  Army’s  ABCS 
(  Army  Battle  Command  System)  6.4  upgrade,  of  which  it  was  reported  that  “Significant 
issues  arose  in  the  [system-of-systems]  SOS  engineering  as  each  program  postured  for 
optimum  solutions  for  its  program”  [Greene  05].  Without  adaptations  to  account  for  the 
dynamic  and  emergent  properties  of  complex  systems  of  systems  (in  this  case,  the  ability  to 
suboptimize  locally  for  the  benefit  of  the  whole),  the  desired  migration  to  network- centric 
operations  is  at  risk. 

A  Q  survey6  of  27  Defense  Information  Systems  Agency  (DISA)  senior  executives 
investigated  the  extent  to  which  they  established  a  shared  understanding  on  where  to  invest 
scarce  resources  for  information  system  initiatives  [Brown  05a].  A  speculative  interpretation 
of  Brown’s  results  suggests  that  traditional  software  engineering  and  the  changing  demands 
of  the  new  network-centric  environment  are  in  conflict. 

Table  1  is  reproduced  from  the  work  by  Brown  and  colleagues  [Brown  05a]  and  is  modified 
to  add  a  column  totaling  the  top  three  priorities  that  DISA  executives  expressed  when  asked 
to  rank  order7  investment  recommendations.  Significant  for  our  purposes  are  the  close 
cumulative  scores  received  by  the  top  four  recommendations: 

1.  conceptualizing  stable  requirements  (53%) 

2.  modeling  tools  to  elaborate  problem-solution  domains  (49%) 

3.  building  and  maintaining  shared  understanding  through  life  cycles  (42%) 

4.  software  applications  (4 1  %) 


6  A  Q  survey  provides  a  means  for  uncovering  stakeholder  perceptions  of  incorrectly  specified 
requirements,  looming  risks,  and  hidden  costs  [Brown  04]. 

7  Survey  respondents  were  asked  to  select  up  to  two  highest  priority  investment  recommendations, 
up  to  three  second  highest,  up  to  four  third  highest,  and  so  on. 
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Table  1:  Frequency  Rates  of  Investment  Priorities 


Investment  Recommendation 

First 

Priority 

(%) 

Second 

Priority 

(%) 

Third 

Priority 

(%) 

Cumulative 

(%) 

Conceptualizing  stable  requirements 

30 

11 

11 

53 

Modeling  tools  to  elaborate  problem-solution 
domains 

19 

26 

4 

49 

Building  and  maintaining  shared 
understanding  through  life  cycles 

4 

19 

19 

42 

Software  applications 

15 

11 

15 

41 

Project  management/oversight  and 
governance  strategies 

11 

11 

4 

26 

Organizational  change  management 

4 

8 

12 

24 

Communication  advances 

0 

4 

19 

23 

Methods  for  tracing  and  tracking  cascading 
costs  of  interdependencies 

8 

8 

4 

20 

Hardware  advances 

7 

7 

0 

14 

The  strong  desire  to  stabilize  requirements,  as  indicated  by  its  top  investment  ranking,  is  the 
bedrock  of  classic  systems  engineering.  Unfortunately,  this  desire  is  at  odds  with  the 
dynamic,  composable,  context-dependent  nature  of  system-of-systems  requirements.  Classic 
requirements  elicitation  can  lead  to  requirements  that  are  specified  in  such  detail  that  concrete 
solutions  can  be  developed  for,  at  best,  some  transient  state  of  a  system  of  systems. 
Alternately,  the  requirement  statements  become  so  vague  as  to  leave  the  solution  space  open 
for  interpretation.  We  believe  that  systems  of  systems  are  better  served  by  a 
capabilities/requirements  expanded  trade  space* * 8  (see  Section  4.2.2)  rather  than  the  symmetric 
demand  structure  of  classic  systems  engineering.  The  expanded  trade  space  is  much  more 
aware  of  user  demands  and  is  reactive  to  asymmetric  demands.9 

The  second-ranked  call  for  modeling  tools  may  indicate  that  DISA  executives  recognize  that 
the  problem  space  is  not  structured  well  enough  to  allow  solutions  that  can  be  drawn  from 
current  knowledge.  This  interpretation  assumes  that  models  are  seen  as  alternatives  to  rigid 
specifications  and  ways  to  characterize  the  feedback  loops  (see  Section  4.3)  and  dynamic 
nature  of  the  complex  problem-solution  space.  We  therefore  interpret  this  ranking  as  a  tacit 
acknowledgement  of  the  ill-structured,  complex  nature  of  the  problem-solution  space. 


‘  Trade  space  is  the  degree  of  flexibility  in  trading  performance  objectives  against  each  other  to 

achieve  best  value,  as  defined  by  the  U.S.  Navy’s  Acquisition  Strategy  Decision  Guide 

(http://www.acquisition.navy.mil/aosfiles/tools/asdg/appendix6.html). 

9  Asymmetric  demands  are  those  demands  that  are  not  satisfied  by  the  existing  supply  mechanisms. 
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According  to  Bardach,  successful  resolution  of  wicked10  or  ill-structured  problems  often 
requires  complex,  cross-cutting  solutions  (i.e.,  solutions  that  require  the  knowledge  and 
expertise  of  several  professional  domains)  [Bardach  98].  The  frequent  use  of  integrated 
product  teams  (IPTs)  is  an  example  of  this  need  for  multiple  professional  domains.  We 
speculate  that  the  survey  respondents’  high  desire  to  build  and  maintain  shared  understanding 
is  indicative  of  the  challenges  associated  with  crossing  these  knowledge  boundaries. 

The  nearly  equivalent  desire  to  invest  in  “software  applications”  may  be  evidence  of  the 
inertia  or  comfort  with  traditional  tools  (i.e.,  let  technology  solve  the  issues,  and  let  the 
applications  fight  it  out).  Unfortunately,  without  fresh  approaches  to  these  complexity  issues, 
traditional  applications  that  are  coded  against  artificially  frozen,  stable  requirement 
specifications  are  doomed  to  failure. 


10  The  nature  of  a  wicked  problem  is  such  that  attempting  to  solve  it  often  reveals  more  complex 
issues.  For  more  information,  see  http://en.wikipedia.org/wiki/Wicked_problems. 
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4  Risks  and  Implications  for  Joint  Capabilities 


Underscoring  the  ways  in  which  traditional  methods  are  in  conflict  with  the  complexities  of 
joint  capabilities,  the  authors’  experience  with  traditional  estimation  tools  has  shown  that 

1.  The  emerging  network-centric,  power-to-the-edge  [Alberts  03], 11  system-of-systems 
solution  space  generates  classes  of  risks  that,  while  evident  in  traditional  systems,  do  not 
appear  to  scale  linearly  with  traditional  measures  such  as  lines  of  code  or  function 
points. 

2.  The  heightened  risk  classes,  along  with  new  military  performance  goals  related  to 
network-centric  operations,12  have  implications  for  the  acquisition  of  systems  of 
systems. 

4.1  Classes  of  Risk  Heightened  in  Joint  Capabilities 

Our  research  into  differentiating  cost  drivers  for  complex,  software-intensive  systems  of 
systems  [Anderson  04]  has  indicated  that  the  following  categories  of  risk  are  much  greater 
for  complex  systems  of  systems  than  they  are  for  their  single-system  counterparts: 

•  missing  requirements 

Missing  requirements  constitute  a  significant  source  of  estimation  error  and  cost  variance 
related  to  system-of-systems  efforts.  Although  missing  requirements  have  always 
troubled  significant  software  systems,  the  issue  escalates  with  each  dimension  of  system- 
of-systems  complexity:  systems,  services,  knowledge  domains,  funding  sources,  users, 
stakeholders,  and  interfaces. 

•  organizational  and  institutional  obstacles 

Joint  teams  suffer  the  additional  complications  of  serving  many  masters.  Each 
stakeholder  commonly  will  have  separate  external  influences — financial,  philosophical,  if 
not  statutory  in  nature.  These  issues  generate  levels  of  inter-  and  intra-team  dynamics  that 
are  exacerbated  by  system-of-systems  efforts. 

•  life-cycle  sustainment 

Life-cycle  sustainment  in  stand-alone  software  systems  is  traditionally  low  risk. 

1 1  Power  to  the  edge  is  a  new  approach  to  command  and  control  proposed  by  Alberts  and  Hayes. 

“ Power  to  the  edge  is  about  changing  the  way  individuals,  organizations,  and  systems  relate  to  one 
another  and  work.  Power  to  the  edge  involves  the  empowerment  of  individuals  at  the  edge  of  an 
organization  (where  the  organization  interacts  with  its  operating  environment  to  have  an  impact  or 
effect  on  that  environment)  or,  in  the  case  of  systems,  edge  devices.  Empowerment  involves 
expanding  access  to  information  and  the  elimination  of  unnecessary  constraints”  [Alberts  03]. 

12  As  described  in  Section  2,  the  performance  goals  now  stress  adaptive  planning,  accelerated 
acquisition  cycles  built  on  spiral  development,  output-based  management,  and  a  reformed  analytic 
support  agenda  [DoD  03b]. 
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However,  the  interdependencies  of  highly  integrated  and  interoperable  systems  generate 
sustainment  issues,  particularly  if  constituent  parts  must  be  maintained  independently. 
Transferring  these  systems  from  development  to  operations  is  more  difficult  in  system- 
of-systems  efforts  due  to  the  need  to  maintain  system-of-systems  interdependencies 
continuously. 

•  team  performance 

Unlike  single-system  development  efforts,  system-of-systems  programs  require 
exceptional  team  performance  in  the  face  of  diverse  team  composition.  Team  members 
often  come  from  disparate  organizations  with  conflicting  goals,  independent  funding,  and 
localized  incentives.  It  takes  tremendous  leadership,  individual  commitment,  and 
flexibility  to  achieve  synergistic  outcomes  in  such  environments. 


4.2  Implications  for  the  Acquisition  Process 

The  implications  of  heightened  system-of-systems  risk  factors  and  new  military  performance 
goals  related  to  network-centric  operations  for  the  acquisition  community  are  unprecedented. 
Because  acquisition  transforms  goals  and  decisions  into  reality,  it  is  the  locus  where  concepts 
become  solidified  into  real-world  tasks  and  operations.  As  such,  we  see  at  least  four 
implications  for  the  acquisition  process. 

4.2.1  Breaching  of  the  Classic  Division  of  Labor 

The  clean,  unambiguous  division  of  labor  that  insulated  acquisition  efforts  in  the  past  will 
have  to  be  breached.  One  reason  for  that  change  is  that  rapid  deployment  needs  will  not  allow 
the  time  needed  to  clarify  all  ambiguity  prior  to  the  acquisition  process.  In  addition,  the 
acquisition  arena  is  not  immune  from  the  complexity  of  joint  capabilities;  within  this  arena, 
we  would  expect  many  hurdles  to  arise  over  requirements  and  battles  to  be  fought  to 
prioritize  different  features  [Slate  02]. 

4.2.2  Complication  of  the  Role  of  Integrated  Architectures 

Integrated  architectures  are  expected  to  provide  the  blueprint  for  where  and  how  operations 
will  intersect  and  overlap  to  provide  joint  capabilities  [Wolfowitz  02].  This  integration  will 
require  acquisition  activities  to  integrate  operations  across  organizations — an  expanded  scope 
that  is  synonymous  with  an  expanded  trade  space.  In  any  system,  and  especially  in  a  system 
of  systems,  quality  attributes  interact:  performance  affects  modifiability;  availability  affects 
safety;  security  affects  performance — and  everything  affects  cost  [Bass  99].  There  is  no 
principled  method  for  characterizing  the  interactions  among  quality  attributes,  and  the  value 
of  these  attributes  will  vary  with  the  specific  situation  [Kazman  00].  System-of-systems 
efforts,  by  nature  of  their  expansiveness,  will  complicate  the  search  for  mutually  acceptable 
solutions  that  meet  joint  requirements  (through  integrated  architectures).  In  fact,  we  expect 
that  the  struggle  over  feature  tradeoffs  will  carry  over  into  the  acquisition  process.  For  all 
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intents  and  purposes,  system-of-systems  efforts  exacerbate  the  struggle  over  competing 
desires. 

4.2.3  Conflicts  Arise  from  Evolutionary  Spiral  Development 

The  need  for  ongoing,  rapid,  and  oftentimes  unanticipated  deployment  requires  the  use  of 
lean  evolutionary  and  spiral  implementation  methods  [Wolfowitz  02].  However,  evolutionary 
acquisition  lacks  clarity  and  thus  makes  the  search  for  solutions  more  dynamic  and  porous 
than  traditional  acquisition  [Sylvester  03].  Slate  stated  that  the  evolutionary  and  spiral 
acquisition  models  make  it  necessary  for  acquirers  to  assume  a  greater  role  in  the 
requirements  process  and  for  “requirers”  (stakeholders  in  the  requirements  process)  to 
assume  a  greater  role  in  the  acquisition  process.  Slate  predicts  that  established  organizational 
relationships  will  be  altered,  and  such  shifts  almost  always  lead  to  conflict  [Slate  02]. 

4.2.4  Competition  Loses  Effectiveness 

Competition  is  traditionally  seen  as  an  effective  means  for  maintaining  a  best-of-breed 
military  [DoD  03b].  This  competitive  dimension  will  do  little  to  arrest  the  struggle  brought 
about  by  joint  capabilities  requirements.  Competitors  tend  to  seek  asymmetric  competitive 
advantage,  not  synergy  and  compromise.  Under  adverse  conditions,  these  struggles  are  likely 
to  express  themselves  in  scope  creep,  schedule  delays,  and  performance  shortfalls. 


4.3  Explicit  Feedback  Loops  Are  Needed 

Even  if  only  some  of  the  above  implications  actually  occur,  it  appears  that  the  acquisition 
process  will  be  held  captive  by  the  effectiveness  of  the  feedback  loops  that  are  established. 
Feedback  loops  will  be  needed  to  clarify  ambiguity  in  and  reduce  friction  among 
interdependencies.  Management  must  make  these  feedback  loops  explicit.  Illuminating, 
monitoring,  and  measuring  such  dynamic  behavior  is  challenging,  but  necessary  to  estimate 
resources  and  budget  properly  for  system-of-systems  acquisition. 

The  DoD  is  attempting  to  leverage  the  benefits  that  systems  of  systems  can  provide  to 
improve  collaboration  efforts  among  the  military  branches,  across  government  agencies,  and 
with  coalition  partners.  The  key  words  presented  in  this  discussion  of  joint  capabilities  (such 
as  spiral,  integrated,  evolutionary,  rapid,  agile,  trade  space,  feedback  loops,  and  tradeoffs) 
indicate  complexity,  and  this  complexity  places  system-of-systems  efforts  at  high  risk  of 
setback  and  failure.  Elucidation  and  illumination  of  the  hidden  threats  of  complexity  may 
provide  the  ability  to  reduce  the  risk  associated  with  these  large-scale  integration  efforts. 
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4.4  Summary  of  Risks  and  Implications 

In  summary,  the  risk  factors  heightened  by  a  system-of-systems  environment  and  the 

military’s  new  performance  goals  challenge  the  acquisition  community  to 

•  breach  the  old  labor  divisions  and  form  dynamic  feedback  loops  between  the  acquirer 
and  the  requirer  as  well  as  among  the  various  autonomous  requirer  constituents 

•  become  more  responsive  and  agile 

•  foster  supplier  competition  and  innovation  that  is  synergistic 

Meanwhile,  the  system-of-systems  engineer  must  do  the  following,  whether  from  the 

acquirer’s  or  requirer’s  perspective: 

•  Learn  to  leverage  a  dynamic  and  expanded  capabilities/requirements  trade  space. 

•  Learn  to  work  in  a  creative  new  solution  space  and  not  always  expect  to  draw  upon 
known  solutions. 

•  Embrace  new  boundary  artifacts, 13  models,  and  simulations  that  promote  shared 
understanding  and  insight  into  the  complexities  of  the  system  of  systems. 


13  A  boundary’  artifact  is  a  mechanism  to  cross  knowledge  domains.  For  example,  a  home’s  blue  print 
is  a  boundary  artifact  that  crosses  the  knowledge  domain  between  the  architect  and  the  home 
owner. 
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5  Solutions  from  Analogous  Domains 


In  response  to  this  paradigm  shift  from  classic  systems  engineering  that  focuses  on  system 
requirements  that  are  stable,  known,  and  able  to  be  specified  in  detail  (well-structured, 
deterministic  systems)  to  an  environment  that  is  driven  to  produce  new  dynamic  capabilities 
among  autonomous  systems  often  characterized  by  interactions  that  evolve  over  time  and 
with  varying  situational  context  (complex,  adaptive  systems),  we  must  look  at  system-of- 
systems  development  with  a  different  lens.  Fortunately,  other  domains  have  studied 
analogous  challenges  and  may  provide  tools  and  techniques  that  can  help. 

Structural  and  dynamic  modeling  tools  and  techniques  have  a  long  history  that  combines  the 
theory,  methods,  and  philosophy  needed  to  analyze  and  influence  the  behavior  of  complex, 
adaptive  systems  [Forrester  91] — in  the  areas  of  management  [Sterman  00,  Weinberg  91], 
environmental  change  [Simonovic  03,  McGrath  01],  strategic  planning  [Lyneis  99,  Stacey 
92],  and  engineering  [Kamopp  00,  Madachy  96],  These  tools  are  commonly  used  to  analyze 
complex  multidimensional  dynamics  of  open,  adaptive  systems  in  order  to  find  nonintuitive 
points  of  leverage  for  improvement  or  for  avoidance  of  nonintuitive  failure  modes. 

However,  these  tools  are  not  frequently  used  to  characterize  problems  and  solutions  for 
software-intensive  systems.  We  have,  however,  observed  the  use  of  system  dynamic  (SD) 
modeling  and  simulation  techniques  that  have  enabled  shared  understanding  of  the  dynamic 
interactions  that  must  take  place  between  people,  organizations,  and  systems  in  specific 
aspects  of  the  acquisition  of  complex  systems  [Adams  05].  Pfahl  and  colleagues  are  doing 
ground  breaking  work  on  the  application  of  SD  modeling  to  software  process  management 
[Pfahl  02].  They  offer  a  method  for  goal-oriented  development  of  SD  models  called 
Integrated  Measurement,  Modeling,  and  Simulation  (IMMoS). 

[IMMoS  offers]  detailed  guidance  in  the  form  of  a  process  model  . . . 
enforces  precise  problem  definition,  helps  to  identify  stakeholders  . . .  defines 
the  product  flow  . . .  provides  templates  and  checklists,  and  offers  hints  on 
when  and  how  to  reuse  information  from  other  software  modeling  activities 
[Pfahl  02]. 

Interestingly,  this  work  is  motivated  not  by  the  complexity  of  system-of-systems  efforts  but 
by  the  general  need  to  improve  software-engineering  decision  support. 

As  evidenced  by  IMMoS,  techniques  to  model  dynamic  system-of-systems  interactions  in  the 
organizations  that  build,  sustain,  use,  and  acquire  these  systems  are  becoming  available.  A 
focused  effort  is  needed  to  adapt  these  techniques  to  software  engineering  and  perhaps  to 
expand  the  boundaries  of  software  engineering  to  encompass  the  organizational  and 
operational  challenges  that  are  critical  for  system-of-systems  success. 
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The  Carnegie  Mellon®  Software  Engineering  Institute  (SEI)  has  recently  committed 
resources  to  analyze  different  approaches  to  structural  and  dynamic  modeling  for  their 
relevance  in  helping  DoD  organizations  to  understand,  develop,  and  acquire  systems  of 
systems  more  effectively.  Our  goals  are  to  understand  the 

•  variety  of  approaches  that  are  being  used  to  implement  these  techniques 

•  conditions  under  which  different  approaches  appear  to  be  feasible  and  successful  in 
meeting  objectives  that  support  successful,  complex,  software-intensive  systems 
acquisition,  production,  and  operation 

•  problem  areas  within  the  software-intensive  systems  problem  space  that  would  be 
amenable  to  solutions  that  involve  some  version  of  system  dynamics  modeling,  emergent 
computation,  or  anticipatory  systems  modeling 

•  adoptability  of  different  approaches  for  different  organizational  settings 

•  value  propositions  associated  with  leveraging  these  tools  and  techniques 


Carnegie  Mellon  is  registered  in  the  U.S.  Patent  and  Trademark  Office  by  Carnegie  Mellon 
University. 


12 


CMU/SEI-2006-TN-029 


6  Conclusions 


As  the  DoD  makes  the  revolutionary  transformation  to  network- centric  operations,  adopting 
new  approaches  for  understanding  and  managing  the  ever-increasing  complexity  of  these 
systems  of  systems  is  critical  for  success.  Common  shared  understanding  of  complex 
contexts,  processes,  and  system  attributes  is  a  key  capability  that  is  needed  to  ensure  shared 
commitment  among  individuals  and  organizations  within  armed  services  units  and  their 
coalition  partners,  as  well  as  among  users,  developers,  and  acquirers  of  the  systems  of 
systems  that  are  being  generated.  The  SEI  is  investigating  and  attempting  to  adapt  existing 
tools  and  techniques  from  domains  that  have  addressed  analogous  complex  issues  into  the 
software  engineering  tool  set.  We  hope  that  doing  this  will  let  us  heed  Galileo’s  advice  and 
make  the  immeasurable  measurable. 

If  you  have  experiences  related  to  the  adoption  or  use  of  the  tools  for  acquisition  discussed  in 
this  technical  note  and  are  willing  to  be  interviewed  as  part  of  our  study,  contact  isis- 
sei@sei.cmu.edu. 
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